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SECTION  I 

INTRODUCTION 

This  report  discusses  developments  in  computer  based  instruction  (CBI) 
and  presents  initiatives  for  the  improvement  of  Navy  instructional  management 
in  the  1985  to  1995  time  frame.  During  the  past  decade  computer  based  instruc¬ 
tion  has  been  widely  applied  in  the  U.S.  Navy.  The  Navy's  Computer  Managed 
Instruction  (CMI)  System,  which  became  operational  in  1973,  provides  a  means 
for  guiding  students  through  a  curriculum  of  instruction.  It  presently  serves 
14  schools  at  five  geographic  locations.  The  approximate  student  average  on 
board  (AOB)  and  throughput  for  CMI  courses  in  FY  78  were  7,000  and  65,000, 
respectively.  A  plan  exists  to  expand  this  system  to  25  schools  and  an  AOB  of 
16,000  students  at  six  locations.  Computer  managed  instruction  is  the  largest 
component  of  CBI  in  the  Naval  Education  and  Training  Command  (Zajkowski, 

Heidt,  Corey,  Mew,  and  Michel i,  1979). 

The  use  of  computers  in  the  management  of  instruction  also  has  been 
instrumental  in  enhancing  the  Navy's  capability  for  the  delivery  of  individ¬ 
ualized  instructions.  In  fact,  the  extent  to  which  instruction  has  been 
individualized  "...  would  have  been  impossible  for  the  number  of  students  we 

are  dealing  with  were  it  not  for  the  availability  of  CMI _  It  gives  a  partial 

answer  to  the  restraints  the  Congress  is  placing  upon  us  for  reductions  in  the 
costs  of  training  and  the  size  of  training  staffs"  (Scanland,  1975). 

All  the  military  services  intend  to  increase  their  use  of  CMI.  Table  1 
shows  the  projections  for  military  CMI  systems  planned  for  the  early  1980s 
(Van  Matre,  1979). 
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TABLE 

1.  MILITARY 

CMI  SYSTEMS 

NAVY 

AIR  FORCE 

ARMY 

MARINE  CORPS 

CMI 

AIS 

CTS 

AIMS 

CBE 

Students  Daily 

16,000 

2,400 

365 

1,600 

2,000 

Courses/Schools 

25 

4 

4 

2 

4-8  CAI 

40  +  CMI 

Locations 

6 

1 

1 

22 

1 

Source:  Van  Matre  (1979) 


At  the  November  1979  meeting  of  the  Chief  of  Naval  Education  and  Training 
(CNET)  Experimental  Training  Programs  Policy  Board  it  was  pointed  out  that 
due  to  technological  advances,  the  current  Navy  CMI  system  will  become  outdated 
in  the  1985  to  1995  time  frame.  The  CNET  tasked  the  Training  Analysis  and 
Evaluation  Group  (TAEG)  to  conduct  a  study  of  the  current  and  projected 
state-of-the-art  in  the  technology  associated  with  the  management  and/or 
delivery  of  instruction  as  a  basis  for  developing  initiatives  for  the  future.1 

PURPOSE  OF  THE  STUDY 

The  present  study  (1)  assesses  the  state-of-the-art  in  instructional 
management  and  instructional  delivery,  (2)  establishes  projections  of  the 
capabilities  for  instructional  management  and  instructional  delivery  systems 
during  the  1985  to  1995  time  frame,  and  (3)  identifies  viable  alternatives  to 
the  current  Navy  CMI  system. 

APPROACH 

There  were  five  major  components  to  the  approach  used  in  this  study. 

First,  an  analysis  was  made  of  current  Navy  CBI  systems  in  terms  of  capabili¬ 
ties  and  problems.  Second,  literature  dealing  with  current  and  future  trends 
of  instructional  management  and  delivery  was  reviewed.  Third,  visits  were 

]CNET  ltr  Code  N-5  of  19  Nov  1979 
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made  to  acknowledged  experts  (listed  in  appendix  A)  for  discussion  and  assess¬ 
ment  of  Navy  needs  and  projected  technological  capabilities  in  hardware, 
software,  and  courseware  to  satisfy  these  needs.  Fourth,  functional  components 
of  computer  based  instructional  management  systems  were  identified  and  analyzed 
in  order  to  arrive  at  the  functions  of  an  ideal  instructional  management 
system.  Finally,  alternative  systems  capable  of  providing  the  functions  were 
identified  and  assessed  for  implementation  constraints  which  would  impact  on 
operational  use  during  the  1985  to  1995  time  frame. 

ORGANIZATION  OF  THIS  REPORT 

In  addition  to  this  introductory  section,  this  report  contains  five 
sections  and  two  appendices.  Section  II  describes  the  major  categories  of 
CBI,  discusses  trends  in  CBI,  and  provides  an  economic  projection  of  CBI 
developments.  Section  III  provides  a  discussion  of  instructional  management 
systems.  Section  IV  provides  the  state-of-the-art  and  trends  of  instructional 
delivery  systems.  Section  V  presents  four  CBI  alternatives  and  contains  an 
assessment  of  these  alternatives  in  terms  of  key  system  characteristics. 

Section  VI  provides  recommendations  for  actions  to  be  taken  in  support  of 
1985  to  1995  CBI  system  development.  Appendix  A  lists  the  consultants  con¬ 
tacted  during  this  study.  Appendix  B  describes  the  major  categories,  sub¬ 
categories,  and  functions  of  CBI. 
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SECTION  II 

CATEGORIES,  TRENDS,  AND  ECONOMIC  ASSESSMENT 
OF  COMPUTER  BASED  INSTRUCTION 

This  section  (1)  identifies  and  defines  the  major  categories  of  CBI  to 
be  analyzed  in  the  remainder  of  the  report,  (2)  discusses  trends  in  CBI,  and 
(3)  provides  economic  projections  of  CBI  developments.  Computer  based  instruc¬ 
tion  is  a  generic  term  used  to  describe  any  instructional  system  in  which 
computer  capabilities  are  utilized  to  manage,  deliver  or  support  instruction, 
or  to  maintain  administrative  information  on  instructional  processes.  This 
definition  does  not  provide  sufficient  specificity  for  detailed  comparative 
analyses  of  various  systems.  However,  a  general  industry  acceptance  of  two 
categories  of  CBI  has  evolved,  computer  assisted/aided  instruction  (CAI)  and 
computer  managed  instruction  (CMI).  The  former  is  involved  in  the  delivery 
of  instruction,  while  the  latter,  in  general,  simply  manages  a  program  of 
instruction  without  a  delivery  component.  Even  though  these  functions  are 
distinctly  different,  the  fact  that  they  can  be  employed  together  in  a  single 
system  has  given  rise  to  some  confusion  in  terms.  Therefore,  an  attempt  is 
made  in  this  report  to  provide  a  framework  for  understanding  and  differentiating 
between  categories  of  CBI. 

MAJOR  CATEGORIES  OF  CBI 

Major  categories  of  CBI  to  be  addressed  in  this  study  are  shown  in 
figure  1.  Computer  based  instructional  management  (CBIM)  includes  those 
functions  which  manage  or  guide  a  student  through  an  instructional  sequence 
such  as  testing,  response  analysis,  prescriptive  guidance,  and  student  mon¬ 
itoring.  Computer  based  instructional  administration  (CBIA)  includes  all  of 
the  record  keeping  and  reporting  required  for  effective  program  administration. 
Computer  based  instruction  support  (CBIS)  includes  instructional/learning 
resource  management  and  allocation,  program  development  coordination,  config¬ 
uration  management,  and  computer  support  functions.  The  functions  included 
in  each  of  these  major  CBI  subcategories  are  shown  in  figure  2.  They  are 
also  described  in  greater  detail  in  appendix  B. 
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TAEG  Report  No.  89 


Computer  based  instructional  delivery  (CBID)  is  a  unique  category  which 
includes  all  computer  based  media  options.  These  options  would  include 
classical  CAI,  computer  graphics,  computer  controlled  video,  computer  simula¬ 
tion,  and  computer  based  training  devices.  Computer  based  instructional 
delivery  together  with  CBIM  has  *  '-most  unlimited  potential  for  development 
and  growth  and  will  therefore  be  the  areas  in  which  Navy  instruction  will 
make  its  greatest  advances.  However,  it  should  be  recognized  that  the  admin¬ 
istration  and  support  categories  of  CBI  (CBIA  and  CBIS)  will  be  essential  for 
the  orderly  and  cost-effective  implementation  of  large  scale  CBI  systems. 

TRENDS 

During  the  next  decade  computer  developments  which  support  instructional 
processes  are  likely  to  evolve  to  the  extent  that  all  functions  which  can  be 
computerized  will  be.  Therefore,  a  key  decision  at  this  time  involves  the 
choice  between  a  system  configuration  which  would  result  from  evolutionary 
growth  of  existing  systems  and  a  large  scale  integrated  system  developed 
under  an  RDT&E  program  in  terms  of  maximum  effectiveness/efficiency  in  the 
1985-1995  time  frame.  However,  a  general  consensus  seems  to  be  that  whatever 
general  system  is  selected  the  underlying  general  development  strategy  should 
include  the  central  management  of  system  and  network  standards  with  autonomous 
decentralized  processing  and  data  base  control.  This  general  strategy  permits 
a  wide  range  of  local  hardware/software  options  for  CBI  while  requiring 
strict  adherence  to  central  management  policy  relating  to  network  configuration, 
data  base  management,  communications  protocol,  and  configuration  management. 

A  number  of  initiatives  will  be  required  to  implement  this  approach 
among  which  will  be  the  phasing  out  of  the  existing  Navy  CMI  System  and  a 
phasing  in  of  a  new  CBI  system  with  greatly  expanded  capability.  A  major 
contributing  factor  to  this  situation  is  an  increasing  trend  toward  the  use 
of  unique  computer  training  applications  arising  out  of  special  training 
requirements  such  as  electronic  warfare  training  and  nuclear  power  training. 
Since  it  is  relatively  certain  that  this  trend  will  continue,  the  instruc¬ 
tional  management  network  of  the  future  must  allow  for  inclusion  of  these 
unique  applications  within  a  global  management  structure.  This  can  only  be 
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accomplished  through  centrally  controlled  policy  development  and  the  estab¬ 
lishment  of  explicit  standards  for  computer  based  instructional  network 
membership. 

Computer  technology  has  developed  to  such  a  point  that  computer  hardware 
cost  has  become  a  secondary  consideration  to  computer  software  cost.  The 
following  paragraphs  address  the  underlying  economic  considerations  which 
have  brought  about  this  change. 

ECONOMIC  ASSESSMENT  OF  CBI 

The  feasibility  of  adopting  new  improvements  in  computer  technology  for 
computer  based  training  applications  depends  upon  both  technical  effective¬ 
ness  and  economic  efficiency.  The  technical  effectiveness,  along  with  some 
insights  into  the  future  course  of  developments  in  instructional  technology, 
is  addressed  elsewhere  in  this  report.  New  and  improved  computer  technology 
which  will  support  effective  training  is  a  necessary  but  not  sufficient 
condition  for  adopting  this  new  technology  for  Naval  training.  The  adoption 
of  any  improved  technology  must  also  be  economically  efficient;  i.e.,  it  must 
also  provide  the  most  economic  means  of  reaching  the  training  objectives  if 
it  is  to  be  rationally  integrated  into  the  Navy  training  system.  Economic 
analysis  is  an  analytical  procedure  for  determining  /he  cost-effectiveness  of 
alternatives  for  satisfying  training  objectives. 

As  previously  mentioned,  this  study  focuses  on  the  anticipated  state  of 
computer  technology  as  it  will  exist  for  the  period  1985-1995  and  is  designed 
to  provide  policy  guidance  as  to  the  appropriate  planning  decisions  which 
will  enable  the  Navy  to  plan  for  the  most  cost-effective,  long-range  training 
system.  Since  the  study  does  not  address  any  specific  training  system  alter¬ 
natives,  it  is  inappropriate  to  provide  an  economic  analysis  supporting  or 
recommending  a  specific  long-range  plan  for  either  accepting  or  rejecting 
projected  computer  technology.  Specific  recommendations  will  depend  on  the 
new  (and  as  yet  undefined)  hardware  systems,  software  requirements,  future 
training  requirements,  existing  resource  configurations,  and  relative  resource 
prices. 


11 


TAEG  Report  No.  89 


TRENDS  IN  COMPUTER  PROCESSING  COSTS.  One  of  the  most  important  considerations 
in  determining  the  economic  desirability  of  adopting  new  and  improved  computer 
technology  is  the  relative  productivity  per  dollar  spent  on  both  the  old  and 
new  system.  Any  training  task  can  be  accomplished  with  an  almost  infinite 
set  of  resource  combinations.  Which  set  is  the  most  economically  efficient 
is  determined  by  both  factor  productivity  and  factor  cost.  Any  disproportionate 
change  in  either  productivity  or  cost  will  ultimately  require  a  change  in  the 
relative  ratios  with  which  the  resources  are  used  in  training.  Often  changes 
in  productivity  and  costs  are  brought  about  by  technological  advances.  In 
computer  technology  there  has  been  a  substantial  increase  in  productivity  and 
both  total  cost  of  hardware  and  average  processing  costs  have  dropped  pre¬ 
cipitously.  These  changes  make  it  imperative  that  the  training  system  be 
reconfigured  in  the  long  run  to  more  fully  utilize  and  capture  the  efficien¬ 
cies  of  the  new  computer  technology. 

The  trend  in  productivity  for  hardware  has  shown  phenomenal  improvement 
over  the  last  several  years.  Most  experts  believe  that  this  rate  of  improve¬ 
ment  will  continue  through  1990.  If  such  improvements  do  continue,  then 
processing  speed  in  1990  is  expected  to  be  approximately  200  times  faster 
than  that  which  existed  in  1970.  This  trend  is  illustrated  in  figure  3. 

The  processing  speed  and  anticipated  reduction  in  real  costs  of  micro¬ 
processing  components  will  contribute  significantly  to  more  cost-effective 
computational  facilities.  Trends  in  costs  of  microprocessor  chips  are  illus¬ 
trated  in  figure  4.  It  is  expected  that  the  cost  of  a  16  bit  microprocessor 
chip  will  cost  less  than  $10  by  1990.  The  cost  of  a  4  bit  microprocessor 
chip  will  be  less  than  $1. 

The  cost  of  memory  is  also  going  down.  If  present  trends  continue,  as 
they  are  expected  to  do,  the  1990  memory  costs  per  bit  will  be  less  than  1 
percent  of  the  1970  costs.  This  trend  is  illustrated  in  figure  5.  The  net 
result  of  improvements  in  microprocessor  costs,  processing  speed,  and  memory 
costs  has  resulted  in  an  extremely  rapid  decline  in  average  storage  and 
processing  costs.  Projections  through  1990  indicate  that  the  average  proces¬ 
sing  costs  will  be  less  than  1  percent  of  the  1970  processing  costs  as  illus¬ 
trated  in  figure  6, 
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INDEX  OF 
PROCESSING 
SPEED 
(1970s  IOO) 


Figure  3.  Index  of  Trend  in  Processing  Speed  Between 
1970  -  1990  (1970  «  100) 


Source:  Adapted  from  Office  of  Technology  Assessment  (1979) 
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(DOLLARS) 


Figure  4.  Projected  Average  Costs  of  Microcomputer  Chips 
(1970  -  1990) 


•J 


urce:  Motorola  Semiconductor  Group  (1980) 


Figure  5- 


Index  of  Trend  in  Memory  Costs  Between 
1970  -  1990 


Source:  Adapted  from  Noyce  (1977) 
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Fiqure  6.  Index  of  Trend  in  Average  Processing  Costs  Between 
1970  -  1990 


Source:  Adapted  from  Office  of  Technology  Assessment  (1979) 


During  the  same  per  km  (in  which  ;  hi  produ<  civi  ty  of  computer  technology 
was  increasing  and  costs  were  dropping-' ,  is.'  average  labor  costs  in  training 
have  been  increasing.  This  h.uige  in  r  iative  :  vice  ratios  shifts  the  economic 
advantage  to  those  alternative  l.  ra  .  n  ir.q  s;.  stem  winch  utilize  computers  to 
substitute  for  the  relatively  expensive  ial.rr. 

SCALE  ECONOMIES.  Scale  econo -lies  r.vi,i  wren  average  training  costs  decrease 
with  increases  in  the  numboi  ot  -  cor  ,  .lined  or  size  of  the  training 
systems.  These  economic,  car  .  '  .r  ■  .  i>.  ran.,  t  ■>  the  opportunity  to 

use  capital  which  is  highly  •.■•••  .  du:  '  recently,  it  was  necessary  to 

use  large  computer  hardware  r.  •  .<  ■  m  (!n  *  ,i  d»  the  computational 

capacity  and  speed  necesset  ■  ■  >:  .  •  ..  ms.  It  nas  simply  not  been 

economically  feasible  to  len.r  a  cm;  ivt  i  .•••■><:  training  system  for  many  of 
the  smaller  training  system  ..  ..  r, •  ■  airing  systems  with  large 

throughput  were  the  only  one-  our.  <>•  tain  •  in:  necessary  efficiencies 

to  justify  a  computerized  ;*\i  die  new  (level opments  in  computer 

technology  have  changed  the  con  .n.  '  inns  in  such  a  way  that  it  will  no 
longer  be  necessary  to  have  ■  a  ,  ;n  mder  to  capture  the  scale 

economies.  The  importance  o i  y  .".pr«\i.  ter-j  m  the  size  of  the  training 
systems  is  becoming  I.  ...  ,r'  tl  ••,.  •  -•♦■sect  ive  calculation. 

SOFTWARE  DLVELOl’Ml  N 1  .  ■.  "•  •  r.  hnii  a  I  capability  of  the 

hardware  has  provided  fro  epee,  • e  ;  ;  c  odputer  'nr  more  sophisticated 

and  widespread  appl  i>  .r  ioi.s.  ,  a  reguire  the  development 

of  unique  software  packages.  ...  ,  v  at  is  extremely  labor-intensive 

and,  therefore,  very  cost'. v.  '  ■  ....  :  use  of  hardware  will  require 

greater  commitment  of  resource  .  to  so'  :iv.rv  >'■,  .  •' inpmont ,  and  in  the  long  run 
software  costs  will  be  the  dominant  .  ns;  • a<  tm  in  the  selection  of  a  com¬ 
puterized  training  system.  Mr,  .  in.  tvasee  ,  est  <  an  he  attributed  to  the 
volume  of  software  whi<  I.  must  <v  v<  loi  e-i  and  not.  to  inefficiencies  in  the 
development  of  software. 

The  total  software  .  t  ir  i  .if  pen  ;  ,m  the  ;iiantity  of  software 

which  must  be  developed  a'  wed  •  •,<  :  bus  v  in  neve  lop  inn  software. 

The  increased  software  fords  pm.  je,  feu  the  future  result  from  the  fact 
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that  with  widespread  use  of  hardware  there  will  be  a  greater  quantity  of 
software  required  for  the  additional  systems.  Tending  to  offset  this  increase 
in  the  cost  of  providing  more  software  will  be  a  decline  in  the  average  cost 
of  developing  the  new  software  packages.  This  decline  in  average  cost  will 
be  due  to  the  development  of  more  sophisticated  high  level  programming  lan¬ 
guages.  These  programming  languages  will  be  highly  user-oriented  and  will 
reduce  labor  requirements  for  developing  specialized  applications  programs. 

The  total  costs  of  developing  and  maintaining  the  high  level  programming 
languages  will  be  high,  but  these  costs  can  be  amortized  over  a  large  number 
of  applications  which  will  tend  to  drive  down  the  average  cost. 

Almost  every  potential  application  of  computer  technology  within  the 
Navy  will  involve  changes  in  training  programs.  The  extent  of  changes  which 
are  economically  efficient  is  one  fundamental  question  which  an  economic 
analysis  must  answer.  A  rational  analysis  would  involve  a  study  of  each  com¬ 
ponent  of  the  existing  training  program  to  determine  which  component  or  module 
can  be  efficiently  computerized.  Since  we  seldom  start  from  "ground  zero," 
there  is  a  need  to  evaluate  the  existing  capital  resource  base  and  determine 
the  most  efficient  means  of  integrating  the  new  computer  technology  into  the 
existing  system.  The  total  replacement  of  sunk  capital  will  seldom  be  econom¬ 
ically  efficient,  especially  in  the  short  run.  However,  in  the  long  run  it 
may  very  well  be  more  efficient  to  plan  for  total  replacement.  Each  problem 
requires  a  specific  analysis,  and  that  analysis  must  be  based  on  an  incremental 
approach. 

The  efficiency  of  each  candidate  system  in  meeting  the  various  management 
and  delivery  functions  will  undoubtedly  differ.  It  would  be  unusual  to  find  a 
single  computerized  system  which  would  be  most  efficient  in  carrying  out  all 
management  and  delivery  functions.  The  computer  system  ultimately  selected 
for  each  application  must  be  one  which  results  from  a  melding  of  the  various 
performance  requirements  with  the  most  cost-effective  computer  system  for 
performing  the  composite  of  those  functions. 
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SECTION  III 

INSTRUCTIONAL  MANAGEMENT  SYSTEMS 

INSTRUCTIONAL  MANAGEMENT  SYSTEM  MODEL 

A  general  model  is  presented  here  relating  general  management  functions 
to  instructional  management  functions.  It  shows  that  the  instructional 
management  process  is  not  unique,  and  it  is  suggested  that  established  manage¬ 
ment  techniques,  when  properly  applied  to  instruction,  can  result  in  greater 
management  efficiency. 

In  spite  of  the  fact  that  state-of-the-art  instructional  management 
systems  are  structured  in  a  variety  of  ways,  they  nevertheless  exhibit  common 
management  characteristics.  These  characteristics  include  planning,  organizing, 
command/executing,  coordinating,  and  controlling  (George,  1972).  A  general 
management  model  with  these  functional  elements  is  represented  diagrammatically 
with  the  addition  of  an  evaluation  function  and  feedback  loop  in  figure  7. 
Without  a  feedback  path,  the  system  would  operate  "open  loop"  and  the  output 
product  could  deviate  significantly  from  plan. 

With  this  general  model,  it  is  possible  to  relate  general  management  to 
instructional  management.  This  relationship  is  presented  diagrammatically  in 
figure  8.  For  the  instructional  management  case,  the  task  becomes  one  of 
managing  a  training  system  in  which  the  learning  process  consists  of  developing 
skills,  knowledges,  and  attitudes.  Entry  testing  provides  a  student  input 
measure;  progress  testing  allows  for  progress  evaluation  and  prescriptive 
guidance  during  the  learning  process;  and  output  testing  certifies  that 
training  objectives  have  been  satisfied.  This  model  can  be  used  to  compare 
specific  types  of  instructional  management  systems  described  in  subsequent 
paragraphs  in  terms  of  the  degree  and  efficiency  with  which  they  are  capable 
of  accomplishing  the  identified  function.  It  is,  however,  not  meant  to  be  a 
comprehensive  model  of  the  instructional  management  process. 
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figure  7.  General  Management  Model 


TAEG  Report  No.  89 


21 


Figure  8.  Instructional  Management  Model 
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Most  current  instructional  management  systems  can  be  categorized  as 
instructor-based  instructional  management  or  computer-based  instructional 
management.  The  instructor-based  case  relates  to  traditional  methods  of 
instructional  management  without  benefit  of  computer  support.  The  computer- 
based  case  relates  to  computer-supported  instructional  management  and  adminis¬ 
trative  data  handling  processes.  The  distinctions  between  these  two  generic 
categories  are  blurred  by  many  possible  types  of  application  of  the  computer 
in  the  instructional  process.  For  example,  an  instructor-managed  system  in 
which  student  training  and  administrative  data  are  stored  in  and  processed  by 
a  computer  might  be  termed  a  form  of  computer-based  management.  However,  in 
this  study,  the  term  computer-based  instructional  management  is  restricted  to 
computer  management  of  instructional  processes  and  not  to  computerized 
administrative,  support,  or  delivery  processes.  The  primary  reason  for  this 
differentiation  is  that  many  existing  CMI  systems  provide  administrative 
information  processing  and  direct  computer-based  delivery  of  instruction. 

These  systems  would  properly  fall  within  the  category  of  computer-based 
instructional  systems  but  not  totally  within  the  subcategory  of  computer- 
based  instructional  management  systems. 

In  this  study,  the  concept  of  computer-based  instructional  management 
will  apply  to  a  range  of  computer  supported  functions.  The  minimal  case  is 
the  use  of  a  computer  for  test  scoring.  This  would  be  similar  to  the  instructor- 
based  management  case  with  the  single  exception  of  computerized  test  scoring. 

At  the  opposite  end  of  the  computer-based  instructional  management  spectrum 
would  be  the  hypothetical  "training  factory"  (Baker  1978)  in  which  all  training 
management  functions  are  computerized.  In  assessing  the  dozen  or  more  instruc¬ 
tional  management  systems  which  are  currently  available,  or  any  other  alterna¬ 
tive  which  might  be  designed,  several  key  issues  must  be  addressed  to  insure 
successful  implementation:  (1)  Are  they  applicable  in  all  training  situations? 

(2)  Are  they  now  or  are  they  projected  to  be  economically  efficient/effective? 

(3)  Are  they  operationally  and  logistically  supportable  over  their  life 
cycle?  (4)  To  what  extent  are  they  subject  to  technological  or  functional 
obsolescence?  (5)  Do  they  have  top  management  support?  (6)  Is  the  user 
community  committed  to  the  implementation  and  maintenance  of  the  system?  (7) 

Is  funding  support  assured  for  the  life  cycle  of  the  system?  Technological 
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considerations,  which  are  well  within  the  state-of-the-art,  do  not  appear  to 
be  a  constraining  factor  for  the  1980  to  1995  time  frame. 

CBIM  SYSTEM  HARDWARE  AND  SOFTWARE  ARCHITECTURE 

Traditionally,  computer-based  instructional  systems  have  consisted  of 
one  or  more  central  processors  communicating  with  a  number  of  time-sharing 
terminals.  This  arrangement  provides  an  interactive  input-output  capability 
which  has  been  applied  for  instructional  management,  instructional  delivery, 
and  administrative  data  handling  functions.  For  the  typical  case,  this 
arrangement  is  still  considered  to  be  a  relatively  direct  and  effective  means 
of  satisfying  computer-based  instructional  system  requirements.  However,  as 
the  application  of  computer-based  instructional  technology  becomes  more 
widespread  and  as  computer  network  architecture  technology  and  capabilities 
evolve,  additional  longer  range  considerations  become  apparent.  It  should 
first  be  determined  if  the  application  of  computer-based  instructional  tech¬ 
nology  in  Navy  training  is  likely  to  increase  during  the  long-range  planning 
period  and,  if  so,  at  what  rate.  The  results  of  this  determination  should 
then  be  used  to  project  the  sequencing  of  integrating  CBI  functions  into 
existing  training  systems.  When  this  instructional  technology  need/capability/ 
implementation  projection  has  been  completed,  system  architectural  alternatives 
can  be  developed.  For  the  case  of  Navy  training,  it  is  envisioned  that  the 
development  and  implementation  of  a  Navy-wide  computer-based  instructional 
management  system  would  be  more  extensive  than  any  computer-based  training 
application  currently  in  use.  Based  upon  an  analysis  of  state-of-the-art 
systems,  the  following  system  characteristics  have  been  identified  as  develop¬ 
ment  guidelines: 

1.  System  hardware  architecture  should  not  reduce  procurement  options 
to  a  single  supplier  for  processing,  storage,  or  communications  components. 

2.  Software  architectural  design  should  follow  state-of-the-art 
thinking  in  relation  to  user  oriented  high  level  language  development, 
structured  programming,  program  modularization,  custom  operating  systems, 
documentation  relevancy,  and  system  wide  software  compatibility. 


23 


TAEG  Report  No.  89 


3.  Distributed  network  architecture  should  provide  a  data  base  design 
which  allows  global  network  data  access  only  where  required.  Local  data  base 
control  should  be  the  general  rule  wherever  practicable.  In  addition, 
standards  for  communication  protocol,  data  handling  software,  message  formats, 
and  data  base  management  should  be  developed  to  preclude  a  dependence  on 
single  supplier  systems.  It  is  highly  probable  that  computer-based  instruc¬ 
tional  management  systems  will  eventually  interface  with  large  numbers  of 
unique  processors  (intelligent  terminals,  special  training  devices,  CAI 
systems,  video  graphic  terminals,  and  videodisc  controllers).  System  commun¬ 
ications  standards  will  be  essential  if  interface  compatibility  is  to  be 
achieved. 

4.  Computer  software  management  systems  together  with  effective  con¬ 
figuration  control  systems  are  essential  for  any  large  scale  computer-based 
development.  Department  of  Defense  (DOD)  software  management  systems  have 
been  evolving  for  a  number  of  years  and  are  now  reasonably  effective  although 
costly  to  implement  and  maintain.  Because  these  systems  are,  in  essence, 
management  control  systems,  they  should  be  carefully  evaluated  in  terms  of 
their  ability  to  maintain  currency  in  an  environment  of  continuous  change. 
Since  a  computer-based  system  of  this  type  becomes  a  management  tool,  it  is 
important  that  control  be  exercised  by  training  program  management. 

In  assessing  current  state-of-the-art  systems,  it  appears  that  the  issue 
again  becomes  one  of  implementation  considerations.  The  hardware  and  soft¬ 
ware  are  available  today  to  mechanize/automate  almost  any  training  function 
envisioned.  Computer  cost  is  decreasing  and  manpower  cost  is  increasing, 
both  at  rates  which  are  likely  to  bring  about  wide  application  of  computer- 
based  training  during  the  next  decade.  Consequently,  it  will  be  necessary  to 
provide  the  needed  planning  and  operational  support  to  assure  operational 
effectiveness  in  the  desired  time  frame. 
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STATE-OF-THE-ART  CBIM  SYSTEMS 

The  application  of  CBIM,  often  referred  to  as  computer  managed  instruc¬ 
tion  (CMI),  has  grown  steadily  during  the  past  decade.  Ten  private  and 
public  sector  applications  were  identified  and  analyzed  in  1974  (Middleton, 
Papetti,  and  Michel i,  1974).  Although  growth  in  the  number  of  systems  has 
not  increased  significantly  since  then,  student  loading  has  continued  to 
increase  and  system  characteristics  have  continued  to  evolve.  The  Navy 
training  CMI  system  is  an  example  of  a  large-scale  centralized  (nationwide 
central  processing)  system  which  was  well  conceived  and  which  continues  to 
perform  effectively.  The  Air  Force  Advanced  Instructional  System  (AIS)  is  an 
example  of  a  decentralized  (base  level  autonomous  operation)  system,  which 
like  the  Versatile  Training  System  (VTS),  the  Aviation  Training  Support 
System  (ATSS),  and  the  Army  Automated  Instructional  Management  System  (AIMS), 
appears  to  have  many  of  the  necessary  CBIM  characteristics.  However,  training 
center  or  school-based  systems  such  as  these  require  extensive  instructional 
program  development,  computer  programming,  and  configuration  management 
support.  These  support  requirements,  combined  with  significant  front-end 
investment,  have  caused  some  extension  in  the  implementation  schedules  of  the 
above  systems.  The  long-term  payoff  still  appears  on  the  horizon,  but  short- 
run  expansion  schedules  are  not  expected  to  meet  original  goals. 

State-of-the-art  military  CMI  systems  have  been  well  described  in  a 
number  of  studies  (Orlansky  and  String,  1979;  Zajkowski,  et  al.,  1979; 
Middleton,  et  al.,  1974)  and  therefore  are  not  described  in  detail  in  this 
report. 

CBIM  TRENDS 

During  the  course  of  this  study,  many  valuable  concepts  relating  to 
future  computer  applications  in  instructional  management  were  offered  by 
experts  in  the  field J  This  expert  opinion  and  the  examination  of  related 

^Since  the  primary  intent  of  this  study  was  to  forecast  developments  in  com¬ 
puter  based  instruction,  emphasis  was  placed  on  discussions  with  experts  in 
this  field.  They  are  named  in  appendix  A. 
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research  led  to  the  development  of  two  major  conclusions.  These  conclusions 
are  presented  below  with  supporting  rationale. 

1.  There  is  a  high  probability  that  the  use  of  CBIM  will  increase 
significantly  during  the  1985  to  1995  time  frame.  The  trends  identified 
which  support  this  position  are: 

a.  decreasing  computer  system  hardware  costs 

b.  increased  software  development  efficiency 

c.  increased  capability  of  applications  software 

d.  wider  user  acceptance  of  computer-based  applications. 

2.  During  the  1985  to  1995  time  frame,  CBIM  will  become  decentralized 
and  highly  autonomous.  However,  strict  adherence  to  central  management 
policy  and  network  standards  will  be  essential  for  a  global  computer  inter- 
conmuni cations  capability  which  appears  to  be  a  necessary  goal  for  system 
management  and  control.  The  trends  identified  which  support  this  conclusion 
are: 


a.  Small,  powerful  computer  systems  will  be  available  at  low  cost 
for  remote  stand-alone  operation. 

b.  Reliability,  maintainability,  supportability,  and  versatility 
figures  of  merit  for  stand-alone  computer  systems  continue  to  improve. 

c.  New  network  architectures  which  allow  for  distributed  data 
base  management,  remote  processing,  and  central  network  control  are  continu¬ 
ing  to  evolve. 


d.  A  strong  trend  toward  stand-alone  CBID,  as  evidenced  by  current 

CAI  and  device  based  training  applications,  will  bring  about  a  requirement 
for  interfacing  these  delivery  systems  with  instructional  management  systems. 
This  can  be  accomplished  most  easily  with  a  highly  decentralized  CBIM  capa¬ 
bility  under  control  of  a  global  network  manager. 
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SECTION  IV 

INSTRUCTIONAL  DELIVERY  SYSTEMS 

This  section  defines  key  terms  in  instructional  delivery,  summarizes 
major  categories  of  instructional  media,  provides  a  brief  analysis  of  selected 
media/delivery  systems,  and  identifies  trends  in  media  development.  In  this 
report  the  term  "media"  will  refer  to  the  means  for  presenting  instructional 
material  to  learners;  for  example,  books,  audiotapes,  and  filmstrips  (see 
CNET  Instruction  1500.12).  The  term  "instructional  delivery  system"  will  refer 
to  all  of  the  elements  with  which  the  student  must  interact  during  the  learn¬ 
ing  process  to  achieve  instructional  goals.  The  structure  of  this  delivery 
system  determines  to  a  major  extent  the  organization  and  presentation  of 
information  pertinent  to  training.  In  other  words,  an  instructional  delivery 
system  is  composed  of  a  student,  a  method  (or  instructional  strategy),  and 
some  form  of  media.  One  of  the  simplest  instructional  delivery  systems  might 
consist  of  a  student  and  printed  instructional  materials,  while  one  of  the 
most  complex  might  combine  the  student  with  an  interactive  adaptive  CAI 
system. 

Factors  that  must  be  considered  in  selecting  media  for  the  instructional 
delivery  system  should  include  the  nature  of  the  tasks,  learning  strategies, 
media  types  available,  costs  of  alternative  media  mixes,  state  of  development 
of  proposed  media  approaches,  and  resources  required  for  courseware  develop¬ 
ment.  TAEG's  Training  Effectiveness  and  Cost  Effectiveness  Prediction  (TECEP) 
technique  (Braby,  1975)  is  a  procedure  for  choosing  instructional  delivery 
systems.  It  is  useful  in  defining  strategies  for  meeting  training  objectives, 
choosing  instructional  delivery  systems  for  carrying  out  these  strategies, 
and  identifying  the  relative  cost  of  the  alternatives.  However,  experience 
with  the  TECEP  technique  has  shown  that  choosing  an  optimum  instructional 
delivery  system  for  military  training  objectives  is  a  subtle  and  complex 
decision  making  task  that  cannot  be  completely  procedural i zed.  Furthermore, 
users  of  the  TECEP  technique  must  possess  expert  knowledge  of  media.  The  use 
of  TECEP,  however,  should  result  in  choosing  the  most  cost-effective  instruc¬ 
tional  delivery  system. 
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CATEGORIES  OF  INSTRUCTIONAL  MEDIA 

Braby  (1975,  table  16)  lists  a  "Media  Pool"  of  89  types  of  instructional 
media  organized  into  seven  categories.  These  categories  are  print  material, 
audio-only  systems,  visual-only  systems,  audio-visual  systems,  CAI/CMI,  simu¬ 
lated  and  operational  systems,  and  special  or  nonstandard  items.  They  are  the 
major  forms  of  instructional  media  being  used  or  being  considered  for  use  in 
military  training  systems.  Examples  from  each  category  are  shown  in  table  2. 
Another  categorization  of  instructional  media  is  shown  in  table  3. 

As  a  matter  of  interest,  table  4  is  included  to  show  how  42  Army  instruc¬ 
tional  media  were  grouped  into  12  "families"  (Silver,  Bennick,  Butler,  and 
Benesch,  1978). 

Videodisc  and  2-D  interactive  computer  graphic  display  systems  have  a  high 
potential  for  innovative  application  in  training.  They  also  have  a  great  deal 
of  power  in  information  transfer  technology  as  well  as  possessing  the  possi¬ 
bility  for  increased  achievement  and  transfer  of  training.  These  systems  will 
be  discussed  in  more  detail  in  later  sections  of  this  report. 

STATE-OF-THE-ART  OF  INSTRUCTIONAL  DELIVERY  SYSTEMS 

The  following  paragraphs  discuss  available  media  in  terms  of  their  capa¬ 
bility  to  improve  instructional  efficiency. 

To  effectively  design  an  instructional  system,  the  training  system 
designer  must  know  the  capabilities  and  limitations  of  various  media  and  their 
related  costs.  A  voluminous  and  detailed  report  by  Rhode,  Esseff,  Pusin, 

Quirk,  and  Shulik  (1970)  has  assessed  the  available  conventional  media  in 
terms  of  their  instructional  flexibility,  support  requirements,  and  initial 
and  operational  costs.  The  reader  is  referred  to  this  report  for  detailed 
information  on  specific  conventional  media.  However,  several  new  learning 
media  are  deserving  of  some  special  attention  because  of  their  high  potential 
in  various  aspects  of  the  delivery  of  instruction. 
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TABLE  2.  CATEGORIES  OF  INSTRUCTIONAL  MEDIA  (After  Braby,  1975) 


Print  Materials: 

books,  self-scoring  exercises,  programmed 
text,  study  card  sets 

Audio-Only  Systems: 

audio  tape  system,  dial  access  information 
retrieval  system,  language  laboratory 

Visual -Only  Systems: 

microform,  panels,  mockup,  slide  projector 
system,  teaching  machine 

Audio-Visual  Systems: 

carrel,  motion  picture,  teaching  machines, 
television  (closed  circuit,  videodisc  and 
video  tape  systems) 

CAI/CMI: 

PLATO  IV,  TICCIT 

Simulated  and  Opera¬ 
tional  Systems: 

on-line  and  off-line  computer  simulations, 
logic  trainers,  operational  equipment 
synthetically  stimulated,  simulators 

Special  and  Non¬ 
standard  Items: 

automatic  raters,  dry  carrel,  classroom 

TABLE  3.  CATEGORIES  OF  INSTRUCTIONAL  MEDIA  (After  Logan,  1977) 
Programmed 

ETV  Radio  Text  CAI/CMI  Audio-Visual 

audio 

audio  tape 
audio  recordings 
audio-visual 
films 
filmstrip 
motion  pictures 
multicolor 
visual 
si  ide 

still  pictures/ 
graphs 
tape/ si ide 
transparencies 
visual 
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TABLE  4.  CATEGORIES  OF  ARMY  INSTRUCTIONAL  MEDIA 
(After  Silver,  et  al.,  1978) 


Job  Materials 

Field  Manuals  (FMs) 
Technical  Manuals  (TMs) 
ITDT  Technical  Manuals 

Printed  Materials 


TEC  Print 

Correspondence  Courses 


Training/ Combat  Literature 


Soldier's  Manual/Job  Book 
Skill  Qualification  Tests  (SQT) 

Army  Training  and  Evaluation  Program  (ARTEP) 

Instructor  with  Standard  Aids 

Class  Packets 
Charts/Display  Boards 
Overhead/Transparencies 
Model s/Mockups 


Audio-Only 

TEC  Audio-Only 
Language  Labs  (GEL) 

Audio-Visual 

TEC  Audio/Visual  (BESELER  CUE/SEE) 
Slides/Sound  Slides 
Army  Training  Films 

Television/Video  Recording 

Classroom  Closed-Circuit  Television  (CCTV) 
Television  Trainer  (TVT) 

Video-Disc 

Computer-Assisted/Managed  Instruction  (CAI/CMI) 

Remote-Access  PLANIT 
PLATO  IV/TUTOR 

ABASCUS  Computerized  Training  System  (CTS) 
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4.  CATEGORIES  OF  ARMY  INSTRUCTIONAL  MEDIA  (continued) 
(After  Silver,  et  al.,  1978) 


Embedded  Training  (ET) 

Operational  TACFIRE  PLANIT 
TACFIRE  Training  System  (TTS) 

TACFIRE  Subsystem  Team  Training  (TSTT) 

Training  Devices/Simulators 

Sand  Table 

Fire  Control  Simulator  BT-33 
Observed  Fire  Trainer  (OFT) 

Artillery  Direct  Fire  Trainer  (ADFT) 

M-31  Field  Artillery  Trainer 

Tactical  Engagement  Simulations  (TES) 

SCOPES 

REALTRAIN  (Exercise  for  Combined  Arms  Elements) 
MILES 

Command/Staff  Battle  Simulations 

Tactical  Exercise  Without  Troops  (TEWTS) 

CPX  Simulation  Facility 

CAMMS  (Computer  Assisted  Map  Maneuver  System) 

Combined  Arms  Tactical  Training  Simulator  (CATTS) 

FIREFIGHT 

DUNN  KEMPF 

PEGASUS 

First  Battle 
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The  current  typical  Navy  delivery  system  is  based  primarily  on  print 
media  moderated  and  managed  by  instructors  (plus  some  use  of  conventional 
audiovisual  media).  This  approach  appears  to  have  reached  its  limit  of  appli¬ 
cations  effectiveness.  Not  only  are  marginal  efficiency  improvements  very 
small  but  usage  is  mismatched  to  the  current  training  need  (Bunderson,  1979a). 

Just  as  conventional  audiovisual  media  have  been  integrated  into  the 
instructional  system  by  matching  them  and  their  specific  advantages  with  the 
objectives  of  training,  it  is  now  essential  to  accomplish  this  for  the  newer 
media;  e.g.,  current  CAI  systems  and  the  other  more  sophisticated  audio-visual 
media  such  as  videodiscs.  Their  successful  use  requires  analysis  of  their 
strengths  and  limitations  through  continuing  research.  For  example,  after  20 
years  of  CAI  system  research  it  has  been  established  that  the  primary  benefit 
of  these  systems  is  derived  from  their  capability  for  individualizing  instruc¬ 
tion.  It  is  capable  of  adjusting  instruction  to  meet  student  performance. 

For  the  current  status  of  CAI,  CMI ,  IMI,  and  PI,  see  Orlansky  and  String 
(1979)  and  Zajkowski,  et  al.  (1979).  Representative  of  the  data  to  be  found 
in  these  reports  are  tables  5  and  6,  reproduced  here  to  show  the  effectiveness 
of  CAI  and  the  extent  to  which  it  is  being  used  operationally  in  Navy  training. 
The  roots  of  computer  applications  for  the  1980's  exist  now,  and  there  is  not 
much  likelihood  that  new  design  concepts  will  change  how  they  are  applied. 
Changes  will  most  likely  be  limited  to  scale  of  use  and  in  who  will  be  using 
them  (Office  of  Technology  Assessment,  1979). 

Two  relatively  new  forms  of  instructional  media  have  great  potential  for 
future  instructional  delivery  systems,  namely,  2-D  interactive  computer 
graphic  display  systems  and  videodisc.  2-0  interactive  computer  graphic 
displays  probably  have  more  potential  for  wide  application  during  the  1985  to 
1995  time  frame  than  any  other  CBID  medium.  This  delivery  mode  can  provide 
visual  2-D  simulations  with  keyboard,  light  pen,  sonic  pen,  photo  sensing  or 
touch  panel  student  interactive  capability.  This  interactive  feature  can  be 
used  in  combination  with  direct  input  3-D  training  components  and  other 
peripheral  equipment  for  a  fully  integrated  training  system  capability. 


COMPUTER-AIDED  INSTRUCTIONAL  PROGRAMS  PLANNED/IN  PLACE  IN  NAVY  TRAINING 
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Because  of  the  computer  dynamic  capability  for  changing  graphic  display 
parameters  (e.g.,  voltage  or  resistance  values  in  electronics  training), 
performance- based  training  will  become  possible.  This  is  currently  being 
accomplished  very  effectively  on  such  systems  as  TICCIT,  PLATO,  and  GETS,  but 
the  cost  is  still  too  high  to  justify  wide  scale  application.  However,  the 
success  of  these  systems  along  with  projections  of  decreasing  costs,  makes 
this  type  of  delivery  very  promising  for  future  applications.  Performance- 
based  training  and  testing  appears  to  be  a  major  design  goal  for  future 
training  systems. 

Videodisc  is  considered  a  revolutionary  development  in  training  tech¬ 
nology  because  of  its  mass  information  storage  capabilities  and  the  flexibil¬ 
ity  in  media  materials  which  can  be  stored.  It  can  store  all  displays  that 
are  now  delivered  on  any  audiovisual  medium,  such  as  color  slides,  super  8  mm 
film,  videotape,  or  motion  picture.  The  108,000  video  pages  of  a  single  disc 
can  be  used  to  deliver  information  in  a  small  fraction  of  the  space  required 
by  print  documentation.  It  integrates  the  features  of  several  different 
media  into  one  simple  storage  and  retrieval  system.  However,  the  authoring 
and  production  of  one  single  master  videotape  of  such  materials  prior  to 
making  master  videodiscs  also  combines  the  problems  of  these  media  (Bunderson 
and  Campbell,  1979). 

Bunderson,  Jarvis,  and  Mendenhall  (1979)  studied  the  implications  for 
the  videodisc  authoring  process  of  the  following  eight  areas:  the  delivery 
system  used  (manually  controlled,  microprocessor  controlled,  computer  con¬ 
trolled  with  either  a  CAI  or  TV  display);  media  selection  during  ISD;  instruc¬ 
tional  strategies;  author  mock-up  and  simulation  prior  to  premastering;  pre¬ 
mastering;  mastering  and  replication;  composition  of  videodisc  authoring 
team;  and  evaluation.  These  researchers  concluded  that  although  optical 
videodisc  technology  and  its  associated  authoring  technologies  can  be  expected 
to  be  undergoing  change  during  the  next  few  years,  videodisc  has  significant 
potential  as  a  new  media  delivery  system  alternative  for  the  Navy.  However, 
they  recommend  that  the  Navy  should  not  plan  to  use  videodiscs  in  the  imme¬ 
diate  future  but  that  the  Naval  Education  and  Training  Command  (NAVEDTRACOM) 
should  track  the  development  of  the  field.  They  further  recommended  that  thp 
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Navy  should  conduct  R&D  involving  the  development  and  mastering  of  a  series 
of  videodisc  strategies  on  promising  delivery  system  alternatives. 

It  is  interesting  to  note  that  Fox  (1980)  believes  that  only  limited 
research  is  needed.  He  believes  it  is  possible  to  demonstrate  new  approaches 
for  instructional  management  and  delivery  systems  by  applying  state-of-the- 
art  instructional  technology  in  existing  classroom  settings.  Fox  (1979)  has 
developed  a  microprocessor-driven  Visual  Image  Controlled  (Vis.  I.  Con)  pro¬ 
jector.  His  stand-alone  instruction  delivery  system  presents  instructional 
materials  on  a  functionally  equivalent  basis  to  that  which  can  be  accomplished 
on  a  video  system.  When  the  instructional  program  is  validated  on  the  Vis.  I. 
Con,  it  can  be  transferred  to  a  videodisc,  which  can  be  controlled  by  a 
microprocessor.  This  microprocessor  can  then  function  as  a  master  controller 
for  both  instructional  management  and  instructional  delivery.  Fox  has  seen 
no  serious  problems  associated  with  current  videodisc  applications  and 
believes  it  is  now  ready  for  use  as  an  operational  component  of  new  instruc¬ 
tional  systems. 

TRENDS 


"Anything  in  the  market  place  in  1988  is  already  in  the  laboratory  in 
1978"  (Fields,  1979).  The  "building  blocks"  are  believed  to  be  known.  What 
will  be  done  in  the  future  is  a  combining  of  the  "blocks"  into  systems  and 
applications. 

According  to  the  Office  of  Technology  Assessment  (1979),  "the  roots  of 
most  new  computer  applications  in  the  1980's  exist  now,  and  there  is  not  much 
likelihood  that  a  new  concept  of  computer  design  will  change  the  way  we  use 
them.  Changes  will  be  in  scale  of  use  and  in  who  will  be  using  them." 

Bunderson  (1979b)  states  that  it  is  not  possible  for  DOD  to  maintain 
conventional  methods  oriented  around  stand-up  instructors  and  printed  materials 
since  training  budgets  are  being  cut  every  year.  New  courseware  forms  and 
new  delivery  systems  must  emerge.  These  new  systems  will  be  dependent  on  the 
widespread  distribution  of  computer  based  delivery  systems. 


A 
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In  the  early  1980's  small  personal  computers  with  limited  memory  will  be 
available  to  provide  answer  analysis  and  feedback,  prompting  help,  and  diag¬ 
nostics.  The  videodisc  will  be  used  increasingly  as  a  medium  for  distribution 
of  instruction.  It  will  become  competitive  with  other  delivery  systems  such 
as  audiocassettes,  videocassettes,  floppy  discs,  and  printed  pages.  Print  and 
audiovisual  media  will  be  transferred  onto  videodiscs  and  microcomputer-based 
systems.  It  will  be  cost  effective  to  transfer  existing  courseware  products 
that  have  been  validated  and  are  widely  used  to  a  single  audiovisual  delivery 
system  such  as  a  videodisc  player. 

The  cost  of  microprocessor  and  memory  module  components  which  provide  the 
computer  capability  for  graphic  display  systems  will  continue  to  decrease  in 
price  for  the  next  decade.  Consequently,  large  scale  application  of  2-D 
interactive  computer  graphic  display  systems  will  be  limited  primarily  by 
software  development  cost.  Even  in  this  relatively  high  cost  area,  many 
software  advances  are  being  made  for  entering  and  storing  graphic  information 
with  low  cost  direct  input  devices.  It  is  probable,  that  within  10  years, 
computer  storage  of  graphic  information  will  be  the  lowest  cost  option  avail¬ 
able  for  storing  training-related  visual  information. 

CAI  will  become  more  common  as  computer-based  delivery  systems  become 
more  widely  used.  The  trend  will  be  away  from  linear  or  fixed  variety  media 
which  just  present  information.  The  trend  will  be  to  provide  interactions 
between  the  trainee  and  media. 

In  addition,  intelligent  courseware,  or  intelligent  CAI  ( I CAI )  will 
increase  rapidly.  Strategies  such  as  the  following  will  be  used:  simulation 
of  a  performance  environment  (e.g.,  troubleshooting  a  circuit),  diagnostic 
model  of  the  trainee's  status  (e.g.,  discrepancies  in  procedural  knowledge), 
prescriptive  model  for  tutoring  a  trainee  (e.g.,  giving  hints  for  trouble¬ 
shooting)  (Bunderson,  1979b). 

In  the  late  1980's  and  beyond,  total  distributed  instructional  delivery 
systems  will  be  available.  The  system  will  be  stand-alone  with  intelligent 
terminals.  The  learning  station  will  have  the  storage  and  display  character- 
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istics  of  videodiscs  so  all  audiovisual  courseware  will  be  displayed  by  a 
single  medium.  CAI  will  be  the  preferred  mode  for  providing  training  with 
feedback  at  a  job  site  or  on-the-job  training,  which,  according  to  Scanland 
(1978),  will  increase  due  to  the  severe  reductions  in  EDTRACOM  resources. 

At  some  duty  stations,  instructors  may  not  be  available,  whereas  in 
schoolhouses  they  will  still  be  essential,  though  their  roles  will  change. 
They  will  function  in  ways  computerized  systems  cannot;  e.g.,  providing  model 
of  the  kind  of  sailor  desired,  stimulating  enthusiasm  and  motivation,  diag¬ 
nosing  problems  not  included  in  the  computerized  system,  and  coordinating 
tasks  that  integrate  skills  learned  with  the  computer. 
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SECTION  V 

INSTRUCTIONAL  MANAGEMENT  ALTERNATIVES 

This  section  describes  instructional  management  alternatives  which  are 
deemed  to  be  viable  in  the  1985-1995  time  frame  and  provides  an  initial  assess¬ 
ment  of  key  management  and  economic  characteristics  for  each.  Two  general 
categories  of  instructional  management  have  been  identified;  namely.  Instructor 
Based  Instructional  Management  (IBIM)  and  Computer  Based  Instructional  Manage¬ 
ment  (CBIM).  Each  of  these  major  categories  can  be  further  refined  into  sub¬ 
categories  by  the  inclusion  of  various  combinations  of  the  features  of  (1) 
centralized  processing  and  data  base  control  vs.  decentralized  processing  and 
data  base  control  and  (2)  the  use  of  existing  technology  vs.  new  technology 
(state-of-the-art  hardware  and  software).  The  eight  possible  combinations  of 
these  features  are  shown  in  table  7.  However,  only  four  of  these  eight  are 
considered  viable  in  the  1985-1995  time  frame.  These  are: 

1.  Instructor  Based  Instructional  Management  (IBIM)--descriptive  of 
traditional  Navy  instruction. 

2.  Centralized  CBIM  using  existing  technology--descriptive  of  the 
existing  Navy  CMI  system. 

3.  Decentralized  CBIM  using  existing  technology--descriptive  of  the 
Navy's  VTS  and  ATSS  and  the  Air  Force's  AIS. 

4.  Decentralized  CBIM  using  new  technology.  This  is  a  reasonable 
description  for  a  1985  to  1995  system  which  would  most  likely  be  developed 
under  an  RDT&E  program. 

Each  alternative  identified  in  table  7  should  be  considered  to  represent 
a  class  of  alternatives.  The  addition  of  some  form  of  CBIA,  CBIS,  and  CBID 
to  each  of  these  major  classes  is  required  to  define  a  specific  computer  based 
training  application.  It  is  the  latter  completely  specified  system  that  will 
provide  the  basis  for  economic  comparisons  of  various  systems.  Until  such 


TABLE  7.  INSTRUCTIONAL  MANAGEMENT  ALTERNATIVES 
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complete  specifications  are  developed  only  the  most  rudimentary  assessment  of 
the  four  major  classes  of  alternatives  is  possible. 

Table  8  serves  to  identify  system  features  and  to  summarize  the  outcome 
of  an  initial  assessment  of  each  of  the  four  classes  of  alternatives  based  on 
these  features.  Each  of  the  evaluative  entries  in  the  cells  of  the  table 
represents  a  consensus  of  the  study  team  based  on  data  collected  during  the 
study.  The  data  for  alternatives  1  and  2  constitute  an  assessment  of  current 
capability  while  the  data  for  alternatives  3  and  4  constitute  predictions  of 
the  system  capabilities. 

Note  that  the  major  characteristics  are  divided  into  two  major  categories— 

management  and  economic.  In  the  former  category  all  management  characteristics 

i 

are  considered  to  be  essential  in  the  1985  to  1995  framework  and  therefore 
should  serve  as  the  basis  for  a  comparative  analysis.  In  the  economic  charac¬ 
teristics,  investment  and  operating  costs,  which  form  the  basis  for  life  cycle 
cost,  combined  with  training  effectiveness  provide  an  estimate  of  life  cycle 
efficiency.  These  are  the  primary  economic  characteristics  which  form  a  basis 
for  a  comparative  analysis.  Expansion  cost  and  efficiency  refer  to  increasing 
the  system  capability  beyond  present  or  designed  capabilities. 

i 
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SECTION  VI 

RECOMMENDATIONS  IN  SUPPORT  OF  1985  TO  1995 
CBI  SYSTEM  DEVELOPMENT 

Recommendations  for  support  of  CBI  system  development  are  presented 
below.  They  are  based  on  study  findings  and  assume  that  a  new  distributed 
computer  system  approach  will  be  recognized  as  the  most  viable  for  future 
applications. 

1.  Establish  a  CNET  staff  management  group  with  single-point  authority 
and  responsibility  for  all  computer  based  instructional  programs. 

2.  Establish  a  management  and  economic  analysis  committee  with  member¬ 
ship  from  CNET,  TAEG,  and  functionals  to  develop  analysis  procedures,  review 
proposals,  and  insure  valid  economic  assessment  of  all  proposed  CBI  applica¬ 
tions.  This  committee  should  be  supportive  of  the  CBI  staff  management  group 
yet  independent  for  purposes  of  management  and  economic  analysis. 

3.  Conduct  an  economic  analysis  to  assess  life-cycle  cost-benefit  for 
each  of  the  four  CBI  alternatives  proposed. 

4.  Determine  the  viable  computer  network  architectural  designs  for 
distributed  processing,  distributed  data  base,  communications  protocol,  and 
network  management  for  proposed  systems. 

5.  Prepare  an  Operational  Requirement  (OR)  to  identify  viable  strategies 
for  integrating  computer  based  instructional  delivery  systems  into  existing 
Navy  training  programs  during  the  1985  to  1995  time  frame. 

6.  Prepare  an  OR  to  determine  the  types  of  software  development  programs 
needed  to  provide  the  necessary  software  support  for  Navy  CBI  applications 
during  the  1985  to  1995  time  frame. 
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7.  Determine  life-cycle  support  requirements  for  large  scale  CBI 
applications  during  the  1985  to  1995  time  frame. 

8.  Prepare  a  master  plan  for  the  continued  utilization  and  support  of 
the  existing  Navy  CMI  system  during  development  and  integration  of  the  1985 
to  1995  time  frame  replacement  system. 
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MAJOR  CATEGORIES,  SUBCATEGORIES  AND  FUNCTIONS  OF  CBI 
COMPUTER  BASED  INSTRUCTIONAL  MANAGEMENT  (CBIM) 

1.  Student  Schedul inq/Sequencinq.  This  functional  category  provides 
for  the  control  and  tracking  of  all  student  activity  beginning  with  registra¬ 
tion  and  ending  with  course  completion  or  precompletion  termination.  It 
utilizes  information  from  student  records,  entry  level  test  results,  course 
organization  and  sequencing  strategy,  progress  and  diagnostic  test  results, 
administrative  inputs,  student  location  control  reports,  time  to  complete 
estimates,  and  time  in  process  measures  as  assignment  alternatives  are  con¬ 
tinuously  assessed  for  optimal  scheduling.  Student  assignment,  scheduling, 
and  tracking  functions  include: 

a.  Student  registration  provides  for  the  initialization  of  the 
student  into  the  training  system.  It  consists  of  entering  student  informa¬ 
tion  into  the  training  system  data  files  to  allow  for  the  maintenance  of 
student  assignment,  progress,  and  history  records.  It  also  provides  student 
profile  information  for  assessment  of  viable  training  track  alternatives  and 
for  estimates  of  course  completion. 

b.  Student  assignment  and  scheduling  includes  the  functions 
required  to  analyze  current  student  status  and  to  generate  student  learning 
guides.  Student  progress  indices,  course  organization  and  mapping,  learning 
resource  availability,  and  progress  test  response  patterns  are  analyzed  to 
provide  a  basis  for  generating  these  guides  which  specify  a  training  path. 

This  process  relates  student  assignment  options  to  student  status  and  learn¬ 
ing  resource  availability  and  then  selects  a  training  path  which  best  satisfies 
the  training  objectives. 

c.  Student  tracking  enters  the  scheduled  activity  into  the  student 
scheduling  and  tracking  program  as  an  in-process  activity  when  a  training  path 
is  selected.  This  activity  is  then  tracked  and  is  utilized  as  a  basis  for 
measuring  student  progress. 
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2.  Module-Lesson  Data  Bank.  The  training  system  module  and  lesson 
data  bank  would  consist  of  a  dedicated  storage  area  within  the  training 
center  computerized  common  data  base  which  would  contain  descriptive  and 
identification  information  for  learning  modules  and  lessons  within  the  train¬ 
ing  center  course  organization  hierarchy.  In  addition,  it  would  contain  a 
description  and  identification  of  the  training  materials,  training  objectives, 
progress  and  diagnostic  test  items  and  audit  vectors  for  instructional 
accountability.  The  functions  within  this  category  include: 

a.  Training  objectives  files  would  fully  describe  and  identify 
the  training  objectives  related  to  course  modules  and  lessons.  These  train¬ 
ing  objectives  would  normally  be  developed  during  a  front  end  analysis  for 
the  Instructional  Program  Development  case.  However,  objectives  could  also 
be  developed  for  existing  programs  of  instruction  through  the  analysis  of 
course  content. 


b.  Training  material  description  and  identification  records  would 
be  developed  for  the  training  material  used  in  support  of  module  or  lesson 
level  instruction.  This  training  material  identification  system  would  be 
used  for  student  guide  generation  related  to  specific  module  and  lesson 
assignment. 


c.  Test  item  files  would  be  grouped  by  module  and  lesson  for 
measuring  student  achievement.  These  testing  materials  would  also  include 
pre,  post,  and  diagnostic  test  items. 

d.  Course  audit  and  accountability  includes  all  of  the  functions 
required  to  assure  the  integrity  of  learning  objectives  as  they  relate  to 
learning  materials  and  test  items. 

3.  Testing.  The  training  system  testing  functional  category  would 
include  methods  for  accessing  all  test  items  to  be  presented  during  the 
testing  process.  Test  items  would  relate  to  the  learning  material  presented 
and  would  follow  predefined  testing  strategies.  Item  generation,  response 
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scoring  and  storage,  item  analysis,  diagnostic  test  item  generation,  random 
ization  of  test  item  sequences,  and  test  security  would  be  included  in  this 
category. 


a.  Test  generation  would  be  possible  for  any  module  or  lesson 
within  a  course  of  instruction.  Selection  and  sequencing  of  test  items  would 
be  generated  in  accordance  with  course  hierarchy  mapping,  learning  module 
identification,  student  achievement,  and  diagnostic  test  results.  Tests 
would  be  generated  in  accordance  with  the  applicable  testing  strategy. 
Uniqueness  resulting  from  randomization  and  customization  would  also  provide 
a  basis  for  developing  the  test  security  system. 

b.  Response  scoring  would  apply  to  the  instructor  scoring  case  or 
the  machine  scoring  case  in  which  a  test  key  is  used  as  a  basis  for  scoring. 
However,  the  computerized  case  would  allow  for  a  decode  of  test  item  random- 
zation  and  would  facilitate  an  analysis  of  test  response  patterns.  Conse¬ 
quently,  greater  prescriptive  guidance  flexibility  would  be  possible  for  the 
computerized  case. 

c.  Training  path  selection  and  prescriptive  guidance  should  be 
possible  for  any  type  of  path  selection  strategy  used  by  the  instructional 
designer.  This  function  is  difficult  and  time  consuming  to  implement  for  the 
instructor  management  case  but  it  is  relatively  easy  to  implement  for  the 
computerized  case. 

d.  Test  response  storage  should  provide  for  short  term  machine 
readable  storage  of  all  student  test  responses.  This  test  response  data  bank 
should  be  converted  to  long  term  machine  readable  storage  when  students 
complete  training.  This  data  can  then  serve  as  a  basis  for  test  item  analysis 
and  course  effectiveness  evaluation. 
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COMPUTER  BASED  INSTRUCTIONAL  ADMINISTRATION  (CBIA) 

1.  Management  and  Administration.  This  category  includes  all  of  the 
organizational  functions  required  to  administer  instructional  programs.  It 
is  intended  to  be  the  master  management  control  category  for  all  other  func¬ 
tional  categories.  The  functions  within  this  category  include  the  following: 

a.  School  management  function  includes  the  planning,  organization, 
and  administration  functions  which  support  school  operations. 

b.  Management  policy  and  control  includes  all  of  the  management 
control  mechanisms  for  assuring  compliance  with  management  policy.  It  relates 
to  the  dissemination  of  management  directives,  to  the  communications  of 
management  information  for  control  structure  feedback,  and  to  the  measurement 
of  management  system  effectiveness. 

c.  Personnel  management  consists  of  the  data  processing  support 
transactions  for  the  administrative  control  of  staff  and  student  personnel. 

d.  Computer  systems  management  includes  all  of  the  organization 
and  management  functions  required  for  the  acquisition,  operation,  and  mainte¬ 
nance  of  computer  hardware  and  software.  Although  the  computer  operations 
function  would  properly  be  located  within  a  computer  group,  all  policy, 
standards,  and  management  control  procedures  should  be  under  the  direction  of 

a  training  directorate  to  assure  compliance  with  training  goals  and  objectives. 

e.  Course  organization  includes  descriptors  and  identifiers  for 
all  modules  and  lessons  within  a  course  hierarchy. 

f.  Student  record  control  includes  all  processes  related  to 
student  record  entries,  information  access,  and  training  transactions. 
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g.  Data  control  and  communications  supports  policy  making  for 
information  standards,  integrity,  and  security.  Although  the  data  management 
system  and  the  report  generation  system  components  would  be  independently 
operable,  data  control  should  remain  within  the  training  system  management 
organization.  Administrative  information  should  be  commi; licated  from  a  single 
data  communications  location  within  a  school  or  training  center  to  any  external 
organization  requiring  the  information.  Student  data,  course  data,  predictive 
data,  and  effectiveness  data  could  be  used  as  a  basis  for  systems  analysis, 
student  assignment,  and  management  control. 

2.  Report  Generation.  This  functional  category  includes  all  of  the 
programs  required  to  access,  convert,  organize,  and  format  data  and  to  gener¬ 
ate  data  output  for  display  or  hard  copy.  It  would  also  include  the  genera¬ 
tion  of  unique  symbols  or  graphic  displays  required  for  special  use  applica¬ 
tions.  The  following  reports  are  examples  of  outputs  from  this  subsystem: 

a.  training  schedules 

b.  administrative  reports 

c.  computer  system  status  reports 

d.  configuration  management  reports 

e.  course/module/lesson  evaluation  reports 

f.  resource  management  reports 

g.  Instructional  Program  Development  status  reports 

h.  productivity/efficiency/cost  reports 

i.  student  learning  guides 

j.  student  progress  reports 

COMPUTER  BASED  INSTRUCTIONAL  SUPPORT  (CBIS) 

1.  Training  Resource  Management.  This  functional  category  relates 

to  the  management  of  training  system  resources  and  should  include  all  per¬ 
sonnel,  facilities,  equipment,  software,  and  expendables  required  during 
execution  of  the  training  process.  As  a  student  transitions  through  a 
course  of  instruction,  student  station  assignment,  learning  materials  assign- 
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ment,  and  instructor  consultation  will  require  scheduling  and  control. 

Although  this  is  currently  accomplished  in  most  Navy  training  settings  by  the 
instructional  staff,  there  are  many  cases  in  which  computer  based  systems 
perform  these  functions. 

a.  Training  resource  inventory  control  includes  all  of  the  functions 
required  to  identify,  catalog,  store,  assign  or  issue,  record,  and  control 

all  physical  training  resources.  Although  personnel  resources  can  be  con¬ 
sidered  training  resources,  they  are  not  included  within  the  resource  inven¬ 
tory  control  function  because  their  control  is  included  in  the  management  and 
administration  function.  A  similar  situation  exists  for  facilities  and 
facilities  support  equipment.  Although  classrooms  and  student  stations  are 
scheduled  by  resource  scheduling  programs,  facilities  managment  is  not 
included  as  a  training  management  function. 

b.  Training  resource  scheduling  includes  the  functional  activity 
related  to  allocating  learning  resources  (such  as  student  stations  and  learn¬ 
ing  materials)  in  such  a  way  as  to  maximize  the  efficiency  of  resource  utili¬ 
zation  and  minimize  the  student  waiting  time  for  resource  assignment.  Sched¬ 
uling  strategies  should  vary  in  accordance  with  course  loading,  resource 
availability,  and  student  assignment  options.  Resource  allocation  records 
should  be  maintained  for  analysis  and  accountability  purposes. 

c.  Training  resource  accountability  functions  include  all  of  the 
activities  relating  to  the  monitoring  of  the  status,  utilization,  and  mainte¬ 
nance  of  resources.  This  function  also  includes  record  keeping  and  analysis 
of  resource  use  to  provide  a  basis  for  adjusting  resource  levels  and  modifying 
resource  scheduling  strategies. 

2.  Instructional  Program  Development  Coordination.  This  functional 
category  includes  all  of  the  activity  required  to  maintain  schedules  for 
learning  material  development  activity.  The  training  material  to  be  devel¬ 
oped  or  under  development  will  normally  add  to  or  replace  existing  training 
resources  and  consequently  will  require  planning  for  integration  into  existing 
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instructional  systems.  The  instructional  management  plan  to  be  utilized  in 
conjunction  with  the  new  and  revised  learning  materials  must  also  be  incor¬ 
porated  into  the  management,  administration,  scheduling,  and  tracking  systems. 

a.  Instructional  program  development  requirements  relates  to  j 

needs  assessment  and  requirements  development.  If  the  changed  or  new  require¬ 
ment  results  in  development  which  will  affect  a  training  program,  a  tracking 
mechanism  should  be  energized  during  the  conceptual  phases.  In  this  way,  the 
research,  development,  and  implementation  process  will  be  coordinated  with 
the  integration  of  the  new  or  revised  training  material  into  the  existing 
training  program  and  the  necessary  planning  and  programing  lead  time  will  be 
provided. 


b.  Program  development  schedule  functions  include  the  planning 
and  scheduling  activity  which  precedes  development.  These  functions  would  be 
carried  out  in  coordination  with  the  development  agent  and  should  result  in  a 
single  development  schedule  which  is  tracked  by  the  developers,  the  imple¬ 
mented,  the  user,  and  the  evaluator. 

c.  Program  development  tracking  and  reporting  relates  to  the 
tracking  activity  which  takes  place  during  the  devleopment  and  implementation 
process.  It  should  be  accompanied  by  information  reporting  to  be  used  by 
instructional  program  development  management  and  by  instructional  process 
management. 

3.  Configuration  Management.  Because  training  materials  and  computer 
programs  are  constantly  in  a  state  of  change  for  the  general  case,  a  need 
exists  to  incorporate  a  configuration  management  system  to  track  changes 
and  configuration  status.  It  is  evident  that  training  objectives  must  relate 
to  training  materials,  while  test  items  must  relate  to  both.  What  is  less 
evident  but  equally  important  is  that  the  course  scheduling  and  tracking 
programs  and  training  resource  allocation  programs  must  also  relate  to  train¬ 
ing  program  objectives.  As  the  process  of  developing  training  systems  becomes 
more  closely  aligned  with  the  development  of  the  more  complex  weapons  systems 
to  which  they  must  necessarily  relate,  a  requirement  to  implement  similar 
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configuration  control  systems  will  become  evident.  To  design,  develop,  and 
implement  some  of  today's  more  sophisticated  computer  based  training  systems 
without  incorporating  a  configuration  management  system  would  be  to  assure 
future  degradation  and  potential  system  failure.  Configuration  management 
techniques  have  been  used  effectively  for  many  years  on  all  types  of  computer 
based  systems  and  it  is  not  envisioned  that  the  application  of  these  tech¬ 
niques  to  computer  based  training  systems  would  cause  any  significant  problems 
Some  of  the  functions  which  should  be  subject  to  configuration  control  include 

a.  course  organization 

b.  facilities  and  training  stations 

c.  training  equipment  and  devices 

d.  module/lesson  objectives  and  test  items 

e.  learning  materials 

f.  computer  hardware 

g.  computer  software. 

Configuration  management  would  provide  the  needed  change  control,  status 
accounting,  and  system  audit  capabilities  for  continuously  evolving  training 
applications.  It  would  allow  for  operational  efficiency  under  conditions  of 
constant  change,  a  characteristic  which  appears  essential  in  today's  dynamic 
environment. 

4.  Computer  Support.  For  those  functional  categories  within  the 
instructional  management  system  which  are  partially  or  fully  computerized 
there  will  be  a  need  to  manage  the  computerization  process.  Both  hardware 
and  software  systems  require  diagnostic  programs  for  preventative  and  correc¬ 
tive  maintenance.  A  full  range  of  maintenance  and  operation  support  documen¬ 
tation  should  be  available  for  life  cycle  support.  This  documentation  should 
include  such  typical  information  as  logic  diagrams,  system  block  diagrams, 
operating  system  descriptions,  application  programs  design  and  interface 
specifications,  data  base  management  organization,  and  data  comnuni cat ions 
requirements  and  specifications.  All  of  this  support  documentation  should  be 
controlled  within  the  configuration  management  functional  category.  An  audit 
capability  should  also  exist  for  validating  the  performance  of  any  hardware 
or  software  subsystem. 
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a.  Computer  hardware  and  software  organization  should  include 
block  diagrams  and  functional  descriptions  for  all  hardware  input,  process¬ 
ing,  storage  and  output  subsystems,  and  the  operating  system  and  applications 
programs.  Data  base  organization,  data  base  access,  and  communications 
protocol  specifications  should  also  be  provided  together  with  operating 
procedures  for  system  use. 

b.  Data  hasp  organization  and  management  documentation  should 
describe  file  organization,  data  formats,  data  types,  data  conversion  require¬ 
ments,  common  data  pool  use,  data  access,  data  protection,  and  report  genera¬ 
tion  interface.  Memory  mapping  for  local  and  shared  data  should  also  be 
provided  for  ease  of  integrating  new  programs  or  modifying  existing  programs. 
Although  much  of  this  information  can  only  be  displayed  in  formats  requiring 
interpretation  by  computer  specialists,  an  attempt  should  be  made  to  describe 
all  data  organization  in  user  language  and  functional  block  diagrams. 

c.  Program  performance  functional  specifications  include  all 
documentation  describing  the  functional  capabilities  of  applications  programs. 
These  functional  descriptions  provide  the  basis  for  validating  program  per¬ 
formance  and  serve  as  the  primary  descriptive  documents  for  establishing 
system  design  goals. 

d.  Interface  specifications  would  maintain  a  current  file  for  all 
intra-computer  and  inter-computer  hardware  and  software  interface  specifica¬ 
tions.  This  information  would  be  available  in  the  form  of  CRT  display  or 
hard  copy  output  and  would  be  maintained  in  accordance  with  configuration 
management  standards. 

e.  Data  communications  protocol  would  maintain  current  files  on  all 
data  communications  standards.  Communications  test  programs  and  network  diag¬ 
nostic  programs  would  also  be  included  for  initialization  and  monitoring  of 
network  operations. 
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f.  System  operating  procedures  would  provide  CRT  display,  hard 
copy  printout  or  operating  manual  description  and  identification  of  all  system 
operating  procedures.  Prompting  and  menu  selection  in  user  terminology  would 
be  provided  for  ease  of  use  by  all  operations  personnel. 

g.  System  maintenance  support  programs  and  documentation  includes 
programming  for  all  maintenance  support  activity.  Maintenance  procedures, 
diagnostics,  references  to  maintenance  documentation,  tools  and  equipment 
listings,  maintenance  action  record  keeping,  maintenance  analysis  reports, 
and  other  software  and  hardware  maintenance  related  programs  would  provide  a 
semi -automated  maintenance  support  capability. 

COMPUTER  BASED  INSTRUCTIONAL  DELIVERY  (CBID) 

1 .  Conventional  CAI 

2.  Computer  Simulation 

3.  Computer  Controlled  Video 

4.  Computer  Graphics. 
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